Electronic apparatus for providing voice recognition control and operating method therefor

ABSTRACT

An example electronic apparatus includes a display; and a processor, wherein the processor may be configured to obtain a content including at least one object; distinguish the at least one object within the content; display an instruction text in correspondence with a non-text object among the at least one object; and select the non-text object corresponding to the instruction text if a voice command corresponding to the instruction text is inputted.

TECHNICAL FIELD

Various embodiments of the present disclosure relate to an electronicdevice providing a voice control function, and an operating methodthereof

BACKGROUND ART

A user interface is technology for providing temporary or continuousaccess to enable communication between a user and an object, a system, adevice, or a program.

In recent years, as a user interface, voice recognition controltechniques for controlling electronic devices using a sound uttered by auser have been studied. A voice recognition control technique may beimplemented by registering instructions to be used for voice control byan application or a service developer in advance and by defining voicerecognition results and actions according to resulting actions.

DETAILED DESCRIPTION OF THE INVENTION Technical Problem

An electronic device employing a voice recognition control technique mayprovide a function or an operation corresponding to a voice command.However, a conventional voice recognition control technique enablescontrol by a voice command established in advance but has difficulty inestablishing data for a document-type content, such as a web page, byassociating a voice command and a functional operation in advance.

Various embodiments of the present disclosure may provide an electronicdevice and an operating method thereof for providing voice recognitioncontrol that enables the selection and execution of a particular objectof content using voice recognition by analyzing a resource of contentconfigured with part of a standardized document displayed on anelectronic device, such as a web page or a GUI toolkit (FEL-EDJE or thelike), and by displaying an object, which is not selectable by voicerecognition, with an assigned instruction text.

Technical Solution

According to various embodiments of the present disclosure, anelectronic device may include: a display; and a processor, wherein theprocessor may be configured to: obtain content including at least oneobject; distinguish the at least one object within the content; displayan instruction text corresponding to a non-text object among the atleast one object; and select the non-text object corresponding to theinstruction text when a voice command corresponding to the instructiontext is input.

According to various embodiments of the present disclosure, an operatingmethod of an electronic device may include: obtaining content includingat least one object; distinguishing the at least one object within thecontent; displaying an instruction text corresponding to a non-textobject among the at least one object; and selecting the non-text objectcorresponding to the instruction text when a voice command correspondingto the instruction text is input.

According to various embodiments of the present disclosure, a computerrecording medium may store an instruction that is executed by at leastone processor and is readable by a computer, wherein the instruction maybe configured to perform: obtaining content including at least oneobject; distinguishing the at least one object within the content;displaying an instruction text corresponding to a non-text object amongthe at least one object; and selecting the non-text object correspondingto the instruction text when a voice command corresponding to theinstruction text is input.

Advantageous Effects

According to various embodiments of the present disclosure, it ispossible to provide voice recognition control that enables the selectionand execution of a particular object using only voice recognition,without any input tool, by analyzing a resource of content configuredwith part of a standardized document, such as a web page or a GUItoolkit (FEL-EDJE or the like), and by displaying an object, which isnot selectable by voice recognition, with an assigned instruction text.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an electronic device 101 in anetwork environment 100 according to various embodiments;

FIG. 2 is a block diagram illustrating an electronic device according tovarious embodiments;

FIG. 3 is a block diagram illustrating a program module according tovarious embodiments;

FIG. 4 is a block diagram illustrating the configuration of anelectronic device according to various embodiments of the presentdisclosure;

FIG. 5 is a flowchart illustrating a method for an electronic device todisplay a content page according to various embodiments of the presentdisclosure;

FIGS. 6A and 6B show examples for illustrating a method for anelectronic device to display a content page according to variousembodiments of the present disclosure;

FIG. 7 is a flowchart illustrating a voice recognition control method ofan electronic device according to various embodiments of the presentdisclosure;

FIGS. 8A and 8B show examples for illustrating a voice recognitioncontrol method of an electronic device according to various embodimentsof the present disclosure;

FIG. 9 is a flowchart illustrating a method in which an electronicdevice controls voice recognition and displays a content page accordingto various embodiments of the present disclosure; and

FIGS. 10A and 10B show examples for illustrating a method in which anelectronic device controls voice recognition and displays a content pageaccording to various embodiments of the present disclosure.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, various embodiments of the present disclosure will bedescribed with reference to the accompanying drawings. However, itshould be understood that there is no intent to limit the presentdisclosure to the particular forms disclosed herein; rather, the presentdisclosure should be construed to cover various modifications,equivalents, and/or alternatives of embodiments of the presentdisclosure. In describing the drawings, similar reference numerals maybe used to designate similar constituent elements.

As used herein, the expression “have”, “may have”, “include”, or “mayinclude” refers to the existence of a corresponding feature (e.g.,numeral, function, operation, or constituent element such as component),and does not exclude one or more additional features.

In the present disclosure, the expression “A or B”, “at least one of Aor/and B”, or “one or more of A or/and B” may include all possiblecombinations of the items listed. For example, the expression “A or B”,“at least one of A and B”, or “at least one of A or B” refers to all of(1) including at least one A, (2) including at least one B, or (3)including all of at least one A and at least one B.

The expression “a first”, “a second”, “the first”, or “the second” usedin various embodiments of the present disclosure may modify variouscomponents regardless of the order and/or the importance but does notlimit the corresponding components. For example, a first user device anda second user device indicate different user devices although both ofthem are user devices. For example, a first element may be termed asecond element, and similarly, a second element may be termed a firstelement without departing from the scope of the present disclosure.

It should be understood that when an element (e.g., first element) isreferred to as being (operatively or communicatively) “connected,” or“coupled,” to another element (e.g., second element), it may be directlyconnected or coupled directly to the other element or any other element(e.g., third element) may be interposer between them. In contrast, itmay be understood that when an element (e.g., first element) is referredto as being “directly connected,” or “directly coupled” to anotherelement (second element), there are no element (e.g., third element)interposed between them.

The expression “configured to” used in the present disclosure may beexchanged with, for example, “suitable for”, “having the capacity to”,“designed to”, “adapted to”, “made to”, or “capable of” according to thesituation. The term “configured to” may not necessarily imply“specifically designed to” in hardware. Alternatively, in somesituations, the expression “device configured to” may mean that thedevice, together with other devices or components, “is able to”. Forexample, the phrase “processor adapted (or configured) to perform A, B,and C” may mean a dedicated processor (e.g., embedded processor) onlyfor performing the corresponding operations or a generic-purposeprocessor (e.g., Central Processing Unit (CPU) or Application Processor(AP)) that can perform the corresponding operations by executing one ormore software programs stored in a memory device.

The terms used herein are merely for the purpose of describingparticular embodiments and are not intended to limit the scope of otherembodiments. A singular expression may include a plural expressionunless they are definitely different in a context. Unless definedotherwise, all terms used herein, including technical and scientificterms, have the same meaning as those commonly understood by a personskilled in the art to which the present disclosure pertains. Such termsas those defined in a generally used dictionary may be interpreted tohave the meanings equal to the contextual meanings in the relevant fieldof art, and are not to be interpreted to have ideal or excessivelyformal meanings unless clearly defined in the present disclosure. Insome cases, even the term defined in the present disclosure should notbe interpreted to exclude embodiments of the present disclosure.

An electronic device according to various embodiments of the presentdisclosure may include at least one of, for example, a smart phone, atablet Personal Computer (PC), a mobile phone, a video phone, anelectronic book reader (e-book reader), a desktop PC, a laptop PC, anetbook computer, a workstation, a server, a Personal Digital Assistant(PDA), a Portable Multimedia Player (PMP), a MPEG-1 audio layer-3 (MP3)player, a mobile medical device, a camera, and a wearable device.According to various embodiments, the wearable device may include atleast one of an accessory type (e.g., a watch, a ring, a bracelet, ananklet, a necklace, a glasses, a contact lens, or a Head-Mounted Device(HMD)), a fabric or clothing integrated type (e.g., an electronicclothing), a body-mounted type (e.g., a skin pad, or tattoo), and abio-implantable type (e.g., an implantable circuit).

According to some embodiments, the electronic device may be a homeappliance. The home appliance may include at least one of, for example,a television, a Digital Video Disk (DVD) player, an audio, arefrigerator, an air conditioner, a vacuum cleaner, an oven, a microwaveoven, a washing machine, an air cleaner, a set-top box, a homeautomation control panel, a security control panel, a TV box (e.g.,Samsung HomeSync™, Apple TV™, or Google TV™), a game console (e.g.,Xbox™ and PlayStation™), an electronic dictionary, an electronic key, acamcorder, and an electronic photo frame.

According to another embodiment, the electronic device may include atleast one of various medical devices (e.g., various portable medicalmeasuring devices (a blood glucose monitoring device, a heart ratemonitoring device, a blood pressure measuring device, a body temperaturemeasuring device, etc.), a Magnetic Resonance Angiography (MRA), aMagnetic Resonance Imaging (MRI), a Computed Tomography (CT) machine,and an ultrasonic machine), a navigation device, a Global PositioningSystem (GPS) receiver, an Event Data Recorder (EDR) , a Flight DataRecorder (FDR) , a Vehicle Infotainment Devices, an electronic devicesfor a ship (e.g., a navigation device for a ship, and a gyro-compass),avionics, security devices, an automotive head unit, a robot for home orindustry, an Automatic Teller's Machine (ATM) in banks, Point Of Sales(POS) in a shop, or internet device of things (e.g., a light bulb,various sensors, electric or gas meter, a sprinkler device, a firealarm, a thermostat, a streetlamp, a toaster, a sporting goods, a hotwater tank, a heater, a boiler, etc.).

According to some embodiments, the electronic device may include atleast one of a part of furniture or a building/structure, an electronicboard, an electronic signature receiving device, a projector, andvarious kinds of measuring instruments (e.g., a water meter, an electricmeter, a gas meter, and a radio wave meter). In various embodiments, theelectronic device may be a combination of one or more of theaforementioned various devices. According to some embodiments, theelectronic device may also be a flexible device. Further, the electronicdevice according to an embodiment of the present disclosure is notlimited to the aforementioned devices, and may include a new electronicdevice according to the development of technology.

Hereinafter, an electronic device according to various embodiments willbe described with reference to the accompanying drawings. In the presentdisclosure, the term “user” may indicate a person using an electronicdevice or a device (e.g., an artificial intelligence electronic device)using an electronic device.

FIG. 1 is a block diagram illustrating an electronic device 101 in anetwork environment 100 according to various embodiments.

Referring to FIG. 1, the electronic device 101 may include a bus 110, aprocessor 120, a memory 130, an input/output interface 150, a display160, and a communication interface 170. In some embodiments, at leastone of the components may be omitted, or additional components may befurther included in the electronic device 101.

The bus 110 may include, for example, a circuit that connects thecomponents to each other and delivers communications (for example,control messages and/or data) between the components.

The processor 120 may include one or more of a Central Processing Unit(CPU), an Application Processor (AP), and a Communication Processor(CP). The processor 120 may control, for example, at least one differentcomponent of the electronic device 101, and/or may perform operationsrelating to communication or data processing.

The memory 130 may include a volatile and/or nonvolatile memory. Thememory 130 may store, for example, a command or data related to at leastone different component of the electronic device 101. According to oneembodiment, the memory 130 may store software and/or a program 140. Theprogram 140 may include, for example, a kernel 141, middleware 143, anApplication Programming Interface (API) 145, and/or an application (or“app”) 147. At least part of the kernel 141, the middleware 143, and theAPI 145 may be designated as an Operating System (OS).

The kernel 141 may control or manage system resources (for example, thebus 110, the processor 120, the memory 130, or the like) used to performan operation or function implemented in other programs (for example, themiddleware 143, the API 145, or the application 147). Further, thekernel 141 may provide an interface that allows the middleware 143, theAPI 145, or the application 147 to access individual components of theelectronic device 101 to thereby control or manage system resources.

The middleware 143 may serve as a relay so that, for example, the API145 or the application 147 communicates with the kernel 141 to exchangedata.

Further, the middleware 143 may process one or more requests foroperations received from the application 147 according to the prioritythereof. For example, the middleware 143 may assign at least oneapplication 147 priority for using system resources of the electronicdevice 101 (for example, the bus 110, the processor 120, the memory 130,or the like). For example, the middleware 143 may process the one ormore requests for operations according to the priority assigned to theat least one application, thereby performing scheduling or loadbalancing for the one or more requests for operations.

The API 145 is, for example, an interface for the application 147 tocontrol a function provided from the kernel 141 or the middleware 143,and may include, for example, at least one interface or function (forexample, a command) for file control, window control, image processing,or text control.

The input/output interface 150 may serve as an interface that delivers acommand or data, which is input from, for example, a user or a differentexternal device, to a different component(s) of the electronic device101. Further, the input/output interface 150 may output a command ordata, which is received from a different component(s) of the electronicdevice 101, to the user or to the different external device.

The display 160 may include, for example, a Liquid Crystal Display(LCD), a Light-Emitting Diode (LED) display, an Organic Light-EmittingDiode (OLED) display, a Micro-Electro-Mechanical Systems (MEMS) display,or an electronic paper display. The display 160 may display, forexample, various types of content (for example, text, an image, a video,an icon, a symbol, or the like) for the user. The display 160 mayinclude a touch screen and may receive touch, gesture, proximity, orhovering input using, for example, an electronic pen or a body part of auser.

The communication interface 170 may establish communication, forexample, between the electronic device 101 and an external device (forexample, a first external electronic device 102, a second externalelectronic device 104, or a server 106). For example, the communicationinterface 170 may be connected to a network 162 via wirelesscommunication or wired communication to communicate with the externaldevice (for example, the second external electronic device 104 or theserver 106).

The wireless communication may use, for example, a cellularcommunication protocol, which may be, for example, at least one ofLong-Term Evolution (LTE), LTE-Advanced (LTE-A), Code Division MultipleAccess (CDMA), Wideband CDMA (WCDMA), Universal MobileTelecommunications System (UMTS), Wireless Broadband (WiBro), and GlobalSystem for Mobile Communications (GSM). Further, the wirelesscommunication may include, for example, short-range communication 164.The short-range communication 164 may include, for example, at least oneof Wireless Fidelity (Wi-Fi), Bluetooth, Near-Field Communication (NFC),and Global Navigation Satellite System (GNSS). The GNSS may include, forexample, at least one of a Global Positioning System (GPS), a GlobalNavigation Satellite System (GNSS), a BeiDou Navigation Satellite System(hereinafter, “BeiDou”), and Galileo, which is the European globalsatellite-based navigation system, depending on the use area orbandwidth. In the present document, “GPS” may be interchangeably usedwith “GNSS” hereinafter. The wired communication may include, forexample, at least one of Universal Serial Bus (USB), High-DefinitionMultimedia Interface (HDMI), Recommended Standard 232 (RS-232), andPlain Old Telephone Service (POTS). The network 162 may include atelecommunications network, which may be, for example, at least one of acomputer network (for example, a Local Area Network (LAN) or Wide AreaNetwork (WAN)), the Internet, and a telephone network.

Each of the first and second external electronic devices 102 and 104 maybe a device of a type that is the same as, or different from, that ofthe electronic device 101. According to one embodiment, the server 106may include a group of one or more servers. According to variousembodiments, all or some of the operations performed in the electronicdevice 101 may be performed in another electronic device or a pluralityof electronic devices (for example, the electronic devices 102 and 104or the server 106). According to one embodiment, when the electronicdevice 101 needs to perform a function or service automatically or uponrequest, the electronic device 101 may request another electronic device(for example, the electronic device 102 or 104, or the server 106) toperform at least some functions related to the function or service,instead of, or in addition to, autonomously performing the function orservice. The other electronic device (for example, the electronic device102 or 104, or the server 106) may perform the requested functions oradditional functions and may transmit the result thereof to theelectronic device 101. The electronic device 101 may provide therequested function or service using the received result without changeor after additionally processing the result. To this end,cloud-computing, distributed-computing, or client-server-computingtechnologies may be used.

FIG. 2 is a block diagram illustrating an electronic device according tovarious embodiments.

The electronic device 201 may include, for example, all or part of theelectronic device 101 illustrated in FIG. 1. The electronic device 201may include one or more processors (for example, Application Processors(APs)) 210, a communication module 220, a Subscriber IdentificationModule (SIM) 224, a memory 230, a sensor module 240, an input device250, a display 260, an interface 270, an audio module 280, a cameramodule 291, a power management module 295, a battery 296, an indicator297, and a motor 298.

The processors 210 may run, for example, an operating system or anapplication to control a plurality of hardware or software componentsthat are connected to the processors 210, and may perform various kindsof data processing and operations. The processors 210 may be configuredas, for example, a System on Chip (SoC). According to one embodiment,the processors 210 may further include a Graphic Processing Unit (GPU)and/or an image signal processor. The processors 210 may include atleast some (for example, a cellular module 221) of the componentsillustrated in FIG. 2. The processors 210 may load a command or datareceived from at least one of other components (for example, nonvolatilememory) into volatile memory to process the command or data, and maystore various types of data in the nonvolatile memory.

The communication module 220 may have a configuration that is the sameas, or similar to, that of the communication interface 170 in FIG. 1.The communication module 220 may include, for example, a cellular module221, a Wi-Fi module 223, a BT module 225, a GNSS module 227, aNear-Field Communication (NFC) module 228, and a Radio Frequency (RF)module 229.

The cellular module 221 may provide, for example, a voice call, a videocall, a text messaging service, or an Internet service through acommunication network. According to one embodiment, the cellular module221 may perform identification and authentication of the electronicdevice 201 in a communication network using a Subscriber Identity Module(SIM, for example, a SIM card) 224. According to one embodiment, thecellular module 221 may perform at least some of the functions providedby the processors 210. According to one embodiment, the cellular module221 may include a Communication Processor (CP).

The Wi-Fi module 223, the BT module 225, the GNSS module 227, and theNFC module 228 may each include a processor to process data transmittedand received via the respective modules. According to one embodiment, atleast some (for example, two or more) of the cellular module 221, theWi-Fi module 223, the BT module 225, the GNSS module 227, and the NFCmodule 228 may be included in one Integrated Chip (IC) or IC package.

The RF module 229 may transmit and receive, for example, a communicationsignal (for example, an RF signal). The RF module 229 may include, forexample, a transceiver, a Power Amplifier (amp) Module (PAM), afrequency filter, a Low-Noise Amplifier (LNA), an antenna, or the like.According to another embodiment, at least one of the cellular module221, the Wi-Fi module 223, the BT module 225, the GNSS module 227, andthe NFC module 228 may transmit and receive an RF signal through aseparate RF module.

The SIM 224 may include, for example, a card including a SIM and/or anembedded SIM, and may include unique identification information (forexample, an Integrated Circuit Card Identifier (ICCID)) or subscriberinformation (for example, an International Mobile Subscriber Identity(IMSI)).

The memory 230 (for example, the memory 130) may include, for example,an internal memory 232 or an external memory 234. The internal memory232 may include, for example, at least one of a volatile memory (forexample, a Dynamic Random-Access Memory (DRAM), a Static RAM (SRAM), aSynchronous Dynamic RAM (SDRAM), or the like) and a nonvolatile memory(for example, a One-Time Programmable Read-Only Memory (OTPROM), aProgrammable ROM (PROM), an Erasable and Programmable ROM (EPROM), anElectrically Erasable and Programmable ROM (EEPROM), a mask ROM, a flashROM, a flash memory (for example, an NAND flash, an NOR flash, or thelike), a hard drive, or a Solid-State Drive (SSD)).

The external memory 234 may further include a flash drive, for example,a Compact Flash (CF), a Secure Digital (SD), a micro Secure Digital(micro-SD), a mini Secure Digital (mini-SD), an extreme digital (xD), amemory stick, or the like. The external memory 234 may be functionallyand/or physically connected to the electronic device 201 through any ofvarious interfaces.

The sensor module 240 may measure, for example, physical quantities, ormay detect the state of operation of the electronic device 201 andconvert measured or detected information into an electrical signal. Thesensor module 240 may include, for example, at least one of a gesturesensor 240A, a gyro sensor 240B, a barometric pressure sensor 240C, amagnetic sensor 240D, an accelerometer 240E, a grip sensor 240F, aproximity sensor 240G, a color sensor 240H (for example, a red, green,and blue (RGB) sensor), a biometric sensor 2401, a temperature/humiditysensor 240J, an illumination sensor 240K, and an ultraviolet (UV) sensor240M. Additionally or alternatively, the sensor module 240 may include,for example, an E-nose sensor, an electromyography (EMG) sensor, anelectroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, aninfrared (IR) sensor, an iris sensor, and/or a fingerprint sensor. Thesensor module 240 may further include a control circuit to control atleast one or more sensors belonging thereto. In one embodiment, theelectronic device 201 may further include a processor configured, as apart of the APs 210 or separately from the APs 210, to control thesensor module 240, thereby controlling the sensor module 240 while theAPs 210 are in a sleep state.

The input device 250 may include, for example, a touch panel 252, a(digital) pen sensor 254, a key 256, or an ultrasonic input device 258.The touch panel 252 may be, for example, at least one of anelectrostatic type, a pressure-sensitive type, an infrared type, and anultrasonic type. Further, the touch panel 252 may further include acontrol circuit. The touch panel 252 may further include a tactile layerto provide a user with a tactile response.

The (digital) pen sensor 254 may, for example, be part of the touchpanel or may include a separate recognition sheet. The key 256 mayinclude, for example, a physical button, an optical key, or a keypad.The ultrasonic input device 258 may detect sound waves via a microphone(for example, a microphone 288) in the electronic device 201 through aninput tool that generates an ultrasonic signal and may identify data.

The display 260 (for example, the display 160) may include a panel 262,a hologram device 264, or a projector 266. The panel 262 may have aconfiguration that is the same as, or similar to, that of the display160 of FIG. 1. The panel 262 may be configured, for example, to beflexible, transparent, or wearable. The panel 262 may be formed with thetouch panel 252 in a single module. The hologram device 264 may displaya three-dimensional image in the air using light interference. Theprojector 266 may project light onto a screen to display an image. Thescreen may be disposed, for example, inside or outside the electronicdevice 201. According to one embodiment, the display 260 may furtherinclude a control circuit to control the panel 262, the hologram device264, or the projector 266.

The interface 270 may include, for example, a High-Definition MultimediaInterface (HDMI) 272, a Universal Serial Bus (USB) 274, an opticalinterface 276, or a D-subminiature (D-sub) interface 278. The interface270 may be included, for example, in the communication interface 170illustrated in FIG. 1. Additionally or alternatively, the interface 270may include, for example, a Mobile High-Definition Link (MHL) interface,a Secure Digital (SD) card/Multi-Media Card (MMC) interface, or anInfrared Data Association (IrDA) interface.

The audio module 280 may bidirectionally convert, for example, a soundand an electrical signal. At least some components of the audio module280 may be included, for example, in the input/output interface 150illustrated in FIG. 1. The audio module 280 may process soundinformation input or output, for example, through a speaker 282, areceiver 284, earphones 286, or the microphone 288.

The camera module 291 is a device that takes, for example, a still imageand a video. According to one embodiment, the camera module 291 mayinclude one or more image sensors (for example, a front sensor or a rearsensor), a lens, an Image Signal Processor (ISP), or a flash (forexample, an LED, a xenon lamp, or the like).

The power management module 295 may manage, for example, the power ofthe electronic device 201. According to one embodiment, the powermanagement module 295 may include a Power Management Integrated Circuit(PMIC), a charger Integrated Circuit (IC), or a battery or fuel gauge.The PMIC may have wired and/or wireless charging methods. The wirelesscharging methods may include, for example, a magnetic resonance method,a magnetic induction method, or an electromagnetic wave method, and mayfurther include an additional circuit for wireless charging, such as acoil loop, a resonance circuit, or a rectifier. The battery gauge maymeasure, for example, the remaining battery charge, the chargingvoltage, the current, or temperature of the battery 296. The battery 296may include, for example, a rechargeable battery and/or a solar battery.

The indicator 297 may display the specific state of the electronicdevice 201 or a component thereof (for example, the processors 210),which may be, for example, a booting state, a message state, or acharging state. The motor 298 may convert an electrical signal intomechanical vibrations, and may generate vibrations or a haptic effect.Although not shown, the electronic device 201 may include a processingdevice for supporting a mobile TV (for example, a GPU). The processingdevice for supporting the mobile TV may process media data in accordancewith Digital Multimedia Broadcasting (DMB), Digital Video Broadcasting(DVB), or mediaFlo™ standards.

Each of the above-described component elements of hardware according tothe present disclosure may be configured with one or more components,and the names of the corresponding component elements may vary based onthe type of electronic device. The electronic device according tovarious embodiments of the present disclosure may include at least oneof the aforementioned elements. Some elements may be omitted or otheradditional elements may be further included in the electronic device.Further, some of the components of the electronic device according tothe various embodiments of the present disclosure may be combined toform a single entity, and thus, may equivalently execute functions ofthe corresponding elements prior to the combination.

FIG. 3 is a block diagram illustrating a program module according tovarious embodiments.

According to one embodiment, the program module 310 (for example, theprogram 140) may include an Operating System (OS) that controlsresources related to an electronic device (for example, the electronicdevice 101) and/or various applications (for example, the application147) that run on the OS. The OS may be, for example, Android, iOS,Windows, Symbian, Tizen, Bada, or the like.

The program module 310 may include a kernel 320, middleware 330, an API360, and/or an application 370. At least part of the program module 310may be preloaded onto the electronic device, or may be downloaded froman external electronic device (for example, the electronic device 102 or104), a server (for example, the server 106), or the like.

The kernel 320 (for example, the kernel 141 of FIG. 1) may include, forexample, a system resource manager 321 and/or a device driver 323. Thesystem resource manager 321 may perform control, allocation, or recoveryof system resources. According to one embodiment, the system resourcemanager 321 may include a process management unit, a memory managementunit, or a file-system management unit. The device driver 323 mayinclude, for example, a display driver, a camera driver, a Bluetoothdriver, a shared memory driver, a USB driver, a keypad driver, a Wi-Fidriver, an audio driver, or an Inter-Process Communication (IPC) driver.

The middleware 330 may provide, for example, functions commonly neededfor applications 370, or may provide the applications 370 with variousfunctions through the API 360 so that the applications 370 mayefficiently use the limited systems resources in the electronic device.According to one embodiment, the middleware 330 (for example, themiddleware 143) may include at least one of a runtime library 335, anapplication manager 341, a window manager 342, a multimedia manager 343,a resource manager 344, a power manager 345, a database manager 346, apackage manager 347, a connectivity manager 348, a notification manager349, a location manager 350, a graphic manager 351, and a securitymanager 352.

The runtime library 335 may include, for example, a library module usedby a complier to add a new function through a programming language whilethe application 370 is running. The runtime library 355 may performfunctions for input/output management, memory management, or arithmeticfunctions.

The application manager 341 may manage, for example, the life cycle ofat least one application among the applications 370. The window manager342 may manage graphic user interface (GUI) resources used for a screen.The multimedia manager 343 may identify formats that are necessary toplay various media files, and may encode or decode a media file using acodec suitable for a corresponding format. The resource manager 344 maymanage resources, such as a source code, a memory, or a storage space,for at least one application among the applications 370.

The power manager 345 may operate with, for example, a BasicInput/Output System (BIOS) to manage a battery or power supply and mayprovide information on power necessary for operation of the electronicdevice. The database manager 346 may generate, retrieve, or change adatabase to be used for at least one application among the applications370. The package manager 347 may install or update an applicationdistributed in the form of a package file.

The connectivity manager 348 may manage wireless connectivity via, forexample, Wi-Fi or Bluetooth. The notification manager 349 may display orreport an incoming message, an appointment, and an event including aproximity notification in a manner that does not disturb a user. Thelocation manager 350 may manage position information on the electronicdevice. The graphic manager 351 may manage a graphic effect to beprovided for the user or a user interface related to the graphic effect.The security manager 352 may provide overall security functionsnecessary for system security or user authentication. According to oneembodiment, when the electronic device (for example, the electronicdevice 101) has phone features, the middleware 330 may further include atelephony manager to manage a voice or video call function of theelectronic device.

The middleware 330 may include a middleware module that formscombinations of various functions of the foregoing components. Themiddleware 330 may provide a specialized module for each type of OS inorder to provide differentiated functions. Further, the middleware 330may dynamically delete some of the existing components or add newcomponents.

The API 360 (for example, the API 145) is, for example, a set of APIprogramming functions, and may be provided with a differentconfiguration depending on the OS. For example, one API set for eachplatform may be provided in the case of Android or iOS, while two ormore API sets for each platform may be provided in the case of Tizen.

The application 370 (for example, the application 147) may include oneor more applications that are capable of providing functions of, forexample, a home screen 371, a dialer 372, SMS/MMS 373, instant messaging(IM) 374, a browser 375, a camera 376, an alarm 377, an address book378, a voice dialer 379, email 380, a calendar 381, a media player 382,an album 383, a clock 384, or health care (for example, for measuringexercising or blood sugar), an environmental data application (forexample, for providing atmospheric pressure, humidity, or temperaturedata), or the like.

According to one embodiment, the application 370 may include anapplication (hereinafter, “information exchange application” forconvenience of description) that supports information exchange betweenthe electronic device (for example, the electronic device 101) and anexternal electronic device (for example, the electronic device 102 or104). The information exchange application may include, for example, anotification relay application for relaying specific information to theexternal electronic device or a device management application formanaging the external electronic device.

For example, the notification relay application may include a functionof relaying notification information, which is generated in anotherapplication (for example, an SMS/MMS application, an email application,a health care application, an environmental data application, or thelike) of the electronic device, to the external electronic device (forexample, the electronic device 102 or 104). Additionally, thenotification relay application may receive notification information, forexample, from the external electronic device and may provide thenotification information to the user.

The device management application may manage (for example, install,delete, or update), for example, at least one function (for example, afunction of turning on/turning off the external electronic device itself(or some components thereof) or adjusting the brightness (or resolution)of a display) of the external electronic device (for example, theelectronic device 102 or 104) communicating with the electronic device,an application operating in the external electronic device, or a service(for example, a call service or message service) provided by theexternal electronic device.

According to one embodiment, the application 370 may include anapplication (for example, a health care application of a mobile medicaldevice) assigned according to the attributes of the external electronicdevice (for example, the electronic device 102 or 104). According to oneembodiment, the application 370 may include an application received fromthe external electronic device (for example, the server 106 or theelectronic device 102 or 104). According to one embodiment, theapplication 370 may include a preloaded application or a third-partyapplication that may be downloaded from a server. The illustratedcomponents of the program module 310, according to the embodiments, maybe termed differently depending on the OS.

According to various embodiments, at least part of the program module310 may be implemented in software, firmware, hardware, or combinationsof at least two or more thereof. At least part of the program module 310may be implemented (for example, run) by, for example, a processor (forexample, the processor 120). At least part of the program module 310 mayinclude, for example, a module, a program, a routine, sets ofinstructions, or a process to perform one or more functions.

The term “module” as used herein may, for example, mean a unit includingone of hardware, software, and firmware or a combination of two or moreof them. The “module” may be interchangeably used with, for example, theterm “unit”, “logic”, “logical block”, “component”, or “circuit”. The“module” may be a minimum unit of an integrated component element or apart thereof. The “module” may be a minimum unit for performing one ormore functions or a part thereof. The “module” may be mechanically orelectronically implemented.

For example, the “module” according to the present disclosure mayinclude at least one of an Application-Specific Integrated Circuit(ASIC) chip, a Field-Programmable Gate Arrays (FPGA), and aprogrammable-logic device for performing operations which has been knownor are to be developed hereinafter.

According to various embodiments, at least some of the devices (forexample, modules or functions thereof) or the method (for example,operations) according to the present disclosure may be implemented by acommand stored in a computer-readable storage medium in a programmingmodule form. The instruction, when executed by a processor (e.g., theprocessor 120), may cause the one or more processors to execute thefunction corresponding to the instruction. The computer-readable storagemedium may, for example, be the memory 130.

The computer readable recoding medium may include a hard disk, a floppydisk, magnetic media (e.g., a magnetic tape), optical media (e.g., aCompact Disc Read Only Memory (CD-ROM) and a Digital Versatile Disc(DVD)), magneto-optical media (e.g., a floptical disk), a hardwaredevice (e.g., a Read Only Memory (ROM), a Random Access Memory (RAM), aflash memory), and the like. In addition, the program instructions mayinclude high class language codes, which can be executed in a computerby using an interpreter, as well as machine codes made by a compiler.The aforementioned hardware device may be configured to operate as oneor more software modules in order to perform the operation of thepresent disclosure, and vice versa.

The programming module according to the present disclosure may includeone or more of the aforementioned components or may further includeother additional components, or some of the aforementioned componentsmay be omitted. Operations executed by a module, a programming module,or other component elements according to various embodiments of thepresent disclosure may be executed sequentially, in parallel,repeatedly, or in a heuristic manner. Furthermore, some operations maybe executed in a different order or may be omitted, or other operationsmay be added. Various embodiments disclosed herein are provided merelyto easily describe technical details of the present disclosure and tohelp the understanding of the present disclosure, and are not intendedto limit the scope of the present disclosure. Accordingly, the scope ofthe present disclosure should be construed as including allmodifications or various other embodiments based on the technical ideaof the present disclosure.

FIG. 4 is a block diagram illustrating the configuration of anelectronic device according to various embodiments of the presentdisclosure.

Referring to FIG. 4, the electronic device 401 according to variousembodiments of the present disclosure may include, for example, theentirety or part of the electronic device 101 illustrated in FIG. 1 andmay include a processor 410, a memory 430, a microphone 450, a display460, and a communication interface 470.

According to various embodiments, the processor 410 may be the processor120 illustrated in FIG. 1. The processor 410 may include one or more ofa Central Processing Unit (CPU), an Application Processor (AP), and aCommunication Processor (CP). The processor 410 may process at leastsome of information obtained from other components of the electronicdevice 401 (for example, the memory 430, the microphone 450, the display450, and the communication interface 470) and may provide theinformation to a user by various methods. According to variousembodiments of the present disclosure, the processor 410 may control allthe components of the electronic device 401.

According to various embodiments, the processor 410 may receive contentincluding at least one object. The content may include a standardizeddocument in at least one of a Hypertext Markup Language (HTML), anExtensible Markup Language (XML), and a GUI toolkit (EFL-EDJE or thelike). The processor 410 may analyze resources of the received content.For example, the processor 410 may analyze the standardized documentincluded in the content and may distinguish at least one objectaccording to the tag type. The at least one object may include at leastone of a content title, an image title, a text icon, an image icon, atext content, an image content, a link content, and an input area whichare displayed on a content page. The processor 410 may also determinewhether the at least one object is related to a text. For example, whenthe object is an object including a text, such as a content title, animage title, a text icon, a text content, and a link content, theprocessor 410 may determine that the object is a text object. When theobject is an object not including a text, such as an image icon, animage content, or an input area, the processor 410 may determine thatthe object is a non-text object.

According to various embodiments, when there is a non-text object thatis not related to a text, the processor 410 may assign an instructiontext to the non-text object. The instruction text may enable thenon-text object to be identified by the user's voice recognition. Inaddition, the processor 410 may configure a content page such that theinstruction text is located corresponding to the object. The processor410 may edit the standardized document in the content so that theinstruction text may be represented on the content page displayed on thedisplay. The instruction text may be displayed in a tooltip format. Inaddition, the processor 410 may control the display 460 to display theconfigured content page.

According to various embodiments, the processor 410 may receive an audiosignal from the microphone 450 and may recognize a voice from thereceived audio signal. For example, the processor 410 may convert theaudio signal from the microphone 450 into a text and may recognize theconverted text. The processor 410 may determine whether the textcorresponding to the recognized voice is related to at least one objectin the content page. When the text corresponding to the recognized voiceis related to the at least one object, the processor 410 may select therelated object. For example, the processor 410 may determine whether atext object including the entirety or at least some of the textcorresponding to the recognized voice is present in the content page.When there is a text object including the text corresponding to therecognized voice, the processor 410 may select the text object as anobject related to the audio signal. Alternatively, the processor 410 maydetermine whether an instruction text including the entirety or at leastsome of the text corresponding to the recognized voice is present in thecontent page. When there is an instruction text including the textcorresponding to the recognized voice, the processor 410 may select anon-text object corresponding to the instruction text as an objectrelated to the audio signal. In addition, when the related object isselected, the processor 410 may perform an operation matching theobject. For example, when the related object is any one of a contenttitle, an image title, a text icon, an image icon, a text content, animage content, and a link content, the processor 410 may access contentcorresponding to the object. Further, when the related object is aninput area, the processor 410 may support voice input according to theselection of the input area.

According to various embodiments, when the text corresponding to therecognized voice is related to a plurality of objects in the contentpage, the processor 410 may assign an index text to the plurality ofrelated objects. The index text may enable the plurality of objects tobe identified by the user's voice recognition. In addition, theprocessor 410 may configure a content page such that the index text islocated corresponding to each object. The processor 410 may edit thestandardized document in the content so that the index text may berepresented on the content page displayed on the display. The index textmay be displayed in a tooltip format. In addition, the processor 410 maycontrol the display 460 to display the configured content page. Invarious embodiments, when the text corresponding to the recognized voiceis related to a plurality of objects in the content page, the processor410 may display an instruction text “cancel” for canceling an audiosignal related to the plurality of objects in a tooltip format on thecontent page displayed on the display so that the user can input anaudio signal including another non-overlapping text.

According to various embodiments, when there is an object related to atext language different from a text language corresponding to therecognized voice, the processor 410 may assign an instruction text tothe object related to the different text language. In addition, theprocessor 410 may configure a content page such that the instructiontext is located corresponding to the object related to the differenttext language, and may control the display 460 to display the configuredcontent page.

According to various embodiments, the memory 430 may be the memory 130illustrated in FIG. 1. The memory 430 may store a command or datarelated to at least one other component of the electronic device 401.The memory 430 may store a signal or data that is input/outputcorresponding to the operations of the processor 410, the microphone450, the display 460, and the communication interface 470 under thecontrol of the processor 410.

According to various embodiments, the microphone 450 may be a part ofthe input/output interface 150 illustrated in FIG. 1. The microphone 450may obtain a voice or sound. The microphone 450 may be always turned onaccording to the operation or may be turned on in response to input ordepending on conditions.

According to various embodiments, the display 460 may be the display 160illustrated in FIG. 1. The display 460 may display a content pageconfigured by the processor 410.

According to various embodiments, the communication interface 470 may bethe communication interface 170 illustrated in FIG. 1. The communicationinterface 470 may connect a session for communication with an externalelectronic device and may communicate data. The communication interface470 may receive content from an external electronic device.

According to various embodiments of the present disclosure, anelectronic device may include: a display; and a processor, wherein theprocessor may be configured to: obtain content including at least oneobject; distinguish the at least one object within the content; displayan instruction text corresponding to a non-text object among the atleast one object; and select the non-text object corresponding to theinstruction text when a voice command corresponding to the instructiontext is input.

According to various embodiments of the present disclosure, theprocessor may be configured to: analyze a resource of the content;distinguish the at least one object within the content on the basis ofan analysis result; assign the instruction text to a non-text objectwhen the at least one object includes the non-text object; and configurea content page such that the instruction text is located correspondingto the non-text object to display the content page on the display.

According to various embodiments of the present disclosure, the contentmay include a standardized document in at least one of a HypertextMarkup Language (HTML), an Extensible Markup Language (XML), and a GUItoolkit (EFL-EDJE).

According to various embodiments of the present disclosure, the at leastone object may include at least one of a content title, an image title,a text icon, an image icon, a text content, an image content, a linkcontent, and an input area which are displayed on the content page.

According to various embodiments of the present disclosure, theinstruction text may be displayed in a tooltip format.

According to various embodiments of the present disclosure, theelectronic device may include a microphone configured to receive anaudio signal from a user, wherein the processor may be configured to:recognize a voice from the audio signal from the microphone; determinewhether a text corresponding to the recognized voice is related to atleast one object in a content page; and select the related object whenthe text is related to the at least one object. Further, the processormay be configured to perform an operation matching the selected object.Further, the processor may be configured to support voice input when theselected object is an input area.

According to various embodiments of the present disclosure, when thetext corresponding to the recognized voice is related to a plurality ofobjects in the content page, the processor may be configured to: assignan index text to the plurality of related objects; and configure acontent page such that the index text is located corresponding to theplurality of related objects to display the content page on the display.

According to various embodiments of the present disclosure, when thereis an object related to a text language different from a text languagecorresponding to the recognized voice, the processor may be configuredto: assign an instruction text to the object related to the differenttext language; and configure a content page such that the instructiontext is located corresponding to the object related to the differenttext language to display the content page on the display.

FIG. 5 is a flowchart illustrating a method for an electronic device todisplay a content page according to various embodiments of the presentdisclosure, and FIGS. 6A and 6B show examples for illustrating a methodfor an electronic device to display a content page according to variousembodiments of the present disclosure.

Hereinafter, a method for an electronic device to display a content pageaccording to various embodiments of the present disclosure will bedescribed with reference to FIGS. 5, 6A, and 6B.

In operation 501, an electronic device (for example, the communicationinterface 470) may receive content from an external electronic device.Alternatively, the electronic device (for example, the processor 410)may receive content stored in the memory 430. The content may include atleast one standardized document of a Hypertext Markup Language (HTML),an Extensible Markup Language (XML), and a GUI toolkit (EFL-EDJE).

In operation 502, the electronic device (for example, the processor 410)may analyze resources of the content. For example, the electronic devicemay analyze the standardized document included in the content and maydistinguish at least one object according to the tag type. The at leastone object may include at least one of a content title, an image title,a text icon, an image icon, a text content, an image content, a linkcontent, and an input area which are displayed on a content page.

In operation 503, the electronic device (for example, the processor 410)may determine whether there is a non-text object among the at least oneobject on the basis of the result of analyzing the resources of thecontent. For example, when the object is an object including a text,such as a content title, an image title, a text icon, a text content,and a link content, the electronic device may determine that the objectis a text object. When the object is an object including no text, suchas an image icon, an image content, and an input area, the electronicdevice may determine that the object is a non-text object.

In operation 504, when there is a non-text object, the electronic device(for example, the processor 410) may assign an instruction text to thenon-text object. The instruction text may enable the non-text object tobe identified by a user's voice recognition.

In operation 505, the electronic device (for example, the processor 410)may configure a content page such that the instruction text is locatedcorresponding to the object. For example, the electronic device may editthe standardized document in the content so that the instruction textmay be represented on the content page displayed on a display. Theinstruction text may be displayed in a tooltip format.

In operation 506, the electronic device (for example, the processor 410)may display the configured content page on the display.

A method for an electronic device to display a content page according tovarious embodiments of the present disclosure may provide a content pagescreen 610 a, as shown in FIG. 6A. The content page screen 610 a may bea web page providing a search function. The electronic device mayanalyze standardized document resources constituting the web page andmay divide the web page according to the tag type into a link object 620that provides an access path to the web page, an input area object 630that provides an input area, an input tool object 640 that provides avirtual keyboard, and a voice search object 650 that provides a voiceinput function. The link object 620, the input area object 630, theinput tool object 640, and the voice search object 650 are objects notincluding any text, and each object may be assigned an instruction text(for example, 1, 2, 3, and 4). For example, on the content page screen610 a illustrated in FIG. 6A, instruction texts (for example, 1, 2, 3,and 4) may be displayed in a tooltip format at positions correspondingto the link object 620, the input area object 630, the input tool object640, and the voice search object 650.

Alternatively, as shown in FIG. 6B, a content page screen 610 b may beprovided. The content page screen 610 b may be a web page that providessearch results. The electronic device may analyze standardized documentresources constituting the web page and may divide the web pageaccording to the tag type into a text object 680 that provides a searchresult title and brief information in a text format and an image object670 that provides an image content. The text object 680 is atext-related object and may not be assigned an instruction text. Theimage object 670 is an object not including any text, and each objectmay be assigned an instruction text (for example, 1, 2, 3, 4, and 5).For example, on the content page screen 610 b illustrated in FIG. 6B, aninstruction text (for example, 1, 2, 3, 4, and 5) may be displayed in atooltip format at a position corresponding to each image object 670.

FIG. 7 is a flowchart illustrating a voice recognition control method ofan electronic device according to various embodiments of the presentdisclosure, and FIGS. 8A and 8B show examples for illustrating a voicerecognition control method of an electronic device according to variousembodiments of the present disclosure.

Hereinafter, a voice recognition control method of an electronic deviceaccording to various embodiments of the present disclosure will bedescribed with reference to FIGS. 7, 8A, and 8B.

In operation 701, an electronic device (for example, the processor 410)may receive an audio signal from the microphone 450. For example, whencontent including at least one object is received and an application(for example, a web browser) that displays the received content on acontent page is executed, the electronic device may provide a voicerecognition control function and may receive an audio signal.Alternatively, when content including at least one object is received,an application (for example, a web browser) that displays the receivedcontent on a content page is executed, and a voice recognition controlfunction is requested by a user, the electronic device may provide thevoice recognition control function in response to the request from theuser and may receive an audio signal. In various embodiments, when thevoice recognition control function is provided, the electronic devicemay analyze resources of the received content and may display a contentpage on which an instruction text is assigned to a non-text object on adisplay.

In operation 702, the electronic device (for example, the processor 410)may recognize a voice from the received audio signal. For example, theelectronic device may convert the audio signal from the microphone intoa text and may recognize the converted text.

In operation 703, the electronic device (for example, the processor 410)may determine whether the text corresponding to the recognized voice isrelated to an object in the content page.

In operation 704, when the text corresponding to the recognized voice isrelated to an object in the content page, the electronic device (forexample, the processor 410) may select the related object.

In operation 705, the electronic device (for example, the processor 410)may perform an operation matching the selected object. For example, whenthe related object is a text object, such as a content title, theelectronic device may access content corresponding to the text object.Alternatively, when the related object is an image object that providesan image content, the electronic device may provide an image contentcorresponding to the image object. Alternatively, when the relatedobject is an input area object, the electronic device may support voiceinput in response to selection of an input area.

A voice recognition control method of an electronic device according tovarious embodiments of the present disclosure may provide a content pagescreen 810 a, as shown in FIG. 8A. The content page screen 810 a may bea web page providing search results. The electronic device may receivean audio signal from the microphone 450 and may recognize a voice fromthe received audio signal. The electronic device may determine whether atext corresponding to the recognized voice is related to at least oneobject in a content page. For example, when the text corresponding tothe recognized voice is “edge”, the electronic device may display 881the selection of an object related to “edge” on the content page screen810 a illustrated in FIG. 8A. Further, the electronic device may performan operation of accessing content of the object simultaneously with theselection of the object related to “edge”.

Alternatively, as illustrated in FIG. 8B, a content page screen 810 bmay be provided. The content page screen 810 b may be a web page thatprovides search results. The electronic device may receive an audiosignal from the microphone 450 and may recognize a voice from thereceived audio signal. The electronic device may determine whether atext corresponding to the recognized voice is related to an instructiontext (for example, 1, 2, 3, 4, and 5 illustrated in FIG. 6B) in acontent page. For example, when the text corresponding to the recognizedvoice is a number “5” on the content page screen 810 b illustrated inFIG. 8B, the electronic device may display 871 the selection of anobject corresponding to an instruction text related to “5”. Further, theelectronic device may perform an operation of providing an image contentof the object simultaneously with the selection of the objectcorresponding to the instruction text related to “5”.

FIG. 9 is a flowchart illustrating a method in which an electronicdevice controls voice recognition and displays a content page accordingto various embodiments of the present disclosure, and FIGS. 10A and 10Bshow examples for illustrating a method in which an electronic devicecontrols voice recognition and displays a content page according tovarious embodiments of the present disclosure.

Hereinafter, a method in which an electronic device controls voicerecognition and displays a content page according to various embodimentsof the present disclosure will be described with reference to FIGS. 9,10A, and 10B.

In operation 901, an electronic device (for example, the processor 410)may receive an audio signal from the microphone 450. For example, whencontent including at least one object is received and an application(for example, a web browser) that displays the received content on acontent page is executed, the electronic device may provide a voicerecognition control function and may receive an audio signal.Alternatively, when content including at least one object is received,an application (for example, a web browser) that displays the receivedcontent on a content page is executed, and a voice recognition controlfunction is requested by a user, the electronic device may provide thevoice recognition control function in response to the request from theuser and may receive an audio signal. In various embodiments, when thevoice recognition control function is provided, the electronic devicemay analyze resources of the received content and may display a contentpage on which an instruction text is assigned to a non-text object on adisplay.

In operation 902, the electronic device (for example, the processor 410)may recognize a voice from the received audio signal. For example, theelectronic device may convert the audio signal from the microphone intoa text and may recognize the converted text.

In operation 903, the electronic device (for example, the processor 410)may determine whether the text corresponding to the recognized voice isrelated to an object in the content page.

In operation 904, the electronic device (for example, the processor 410)may determine whether the text corresponding to the recognized voice isrelated to a plurality of objects in the content page.

In operation 905, when the text corresponding to the recognized voice isrelated to one object, the electronic device (for example, the processor410) may select the related object.

In operation 906, the electronic device (for example, the processor 410)may perform an operation matching the selected object.

In operation 907, when the text corresponding to the recognized voice isrelated to a plurality of objects, the electronic device (for example,the processor 410) may assign an index text to the plurality of relatedobjects. The index text may enable the plurality of objects to beidentified by the user's voice recognition.

In operation 908, the electronic device (for example, the processor 410)may configure a content page such that the index text is locatedcorresponding to each object. For example, the electronic device mayedit a standardized document of the content so that the index text maybe represented on the content page displayed on the display. The indextext may be displayed in a tooltip format.

In operation 909, the electronic device (for example, the processor 410)may display the configured content page on the display.

A method in which an electronic device controls voice recognition anddisplays a content page according to various embodiments of the presentdisclosure may provide a content page screen 1010 a, as shown in FIG.10A. The content page screen 1010 a may be a web page providing searchresults. The electronic device may receive an audio signal from themicrophone 450 and may recognize a voice from the received audio signal.The electronic device may determine whether a text corresponding to therecognized voice is related to a plurality of objects in a content page.For example, when the text corresponding to the recognized voice is“galaxy”, the electronic device may display 1090 the selection of aplurality of objects related to “galaxy” on the content page screen 1010a illustrated in FIG. 10A.

As illustrated in FIG. 10B, on a content page screen 1010 b, an indextext (for example, 1, 2, 3, 4, 5, 6, and 7) may be assigned to each ofthe plurality of objects 1090 related to “galaxy”, and the index text(for example, 1, 2, 3, 4, 5, 6, and 7) may be displayed in a tooltipformat at a position corresponding to each object.

Further, as illustrated in FIG. 10B, on the content page screen 1010 b,an instruction text “cancel” 1095 for canceling an audio signal relatedto the plurality of objects 1090 may be displayed in a tooltip format onthe content page displayed on the display so that the user can input anaudio signal including another non-overlapping text.

In addition, as illustrated in FIG. 10B, when there is an object relatedto a text language (for example, Korean) different from a text language(for example, English) corresponding to the recognized voice, the objectis not selected, in which case the electronic device may assign a newinstruction text to the object related to the different text language.

According to various embodiments of the present disclosure, an operatingmethod of an electronic device may include: obtaining content includingat least one object; distinguishing the at least one object within thecontent; displaying an instruction text corresponding to a non-textobject among the at least one object; and selecting the non-text objectcorresponding to the instruction text when a voice command correspondingto the instruction text is input.

According to various embodiments of the present disclosure, theoperating method may further include: analyzing a resource of thecontent; distinguishing the at least one object within the content onthe basis of an analysis result; assigning the instruction text to anon-text object when the at least one object includes the non-textobject; and configuring and displaying a content page such that theinstruction text is located corresponding to the non-text object.

According to various embodiments of the present disclosure, the contentmay include a standardized document in at least one of a HypertextMarkup Language (HTML), an Extensible Markup Language (XML), and a GUItoolkit.

According to various embodiments of the present disclosure, the at leastone object may include at least one of a content title, an image title,a text icon, an image icon, a text content, an image content, a linkcontent, and an input area which are displayed on the content page.

According to various embodiments of the present disclosure, theinstruction text may be displayed in a tooltip format.

According to various embodiments of the present disclosure, theoperating method may further include: receiving an audio signal from auser; recognizing a voice from the received audio signal; determiningwhether a text corresponding to the recognized voice is related to atleast one object in a content page; and selecting the related objectwhen the text is related to the at least one object. Further, theoperating method may further include performing an operation matchingthe selected object. In addition, when the selected object is an inputarea, the operating method may further include supporting voice input.

According to various embodiments of the present disclosure, when thetext corresponding to the recognized voice is related to a plurality ofobjects in the content page, the operating method may further includeassigning an index text to the plurality of related objects, andconfiguring and displaying a content page such that the index text islocated corresponding to the plurality of related objects.

According to various embodiments of the present disclosure, when thereis an object related to a text language different from a text languagecorresponding to the recognized voice, the operating method may furtherinclude assigning an instruction text to the object related to thedifferent text language, and configuring and displaying a content pagesuch that the instruction text is located corresponding to the objectrelated to the different text language.

According to various embodiments of the present disclosure, a computerrecording medium may store an instruction that is executed by at leastone processor and is readable by a computer, wherein the instruction maybe configured to perform: obtaining content including at least oneobject; distinguishing the at least one object within the content;displaying an instruction text corresponding to a non-text object amongthe at least one object; and selecting the non-text object correspondingto the instruction text when a voice command corresponding to theinstruction text is input.

Embodiments of the present disclosure illustrated in the presentspecification and the drawings are provided merely to easily describetechnical details of the present disclosure and to help theunderstanding of the present disclosure, and are not intended to limitthe scope of the present disclosure. Accordingly, the scope of thepresent disclosure should be construed as including all modifications orvarious other embodiments based on the technical idea of the presentdisclosure in addition to the embodiments disclosed herein.

Industrial Applicability

According to various embodiments of the present disclosure, it ispossible to provide voice recognition control that enables the selectionand execution of a particular object only using voice recognition,without any input tool, by analyzing a resource of content configuredwith part of a standardized document, such as a web page or a GUItoolkit (FEL-EDJE or the like), and by displaying an object, which isnot selectable by voice recognition, with an assigned instruction text.

1. An electronic device comprising: a display; and a processor, whereinthe processor is configured to: obtain content comprising at least oneobject; distinguish the at least one object within the content; displayan instruction text corresponding to a non-text object among the atleast one object; and select the non-text object corresponding to theinstruction text when a voice command corresponding to the instructiontext is input.
 2. The electronic device of claim 1, wherein theprocessor is configured to: analyze a resource of the content;distinguish the at least one object within the content on the basis ofan analysis result; assign the instruction text to a non-text objectwhen the at least one object comprises the non-text object; andconfigure a content page such that the instruction text is locatedcorresponding to the non-text object to display the content page on thedisplay.
 3. The electronic device of claim 1, wherein the contentcomprises a standardized document in at least one of a Hypertext MarkupLanguage (HTML), an Extensible Markup Language (XML), and a GUI toolkit.4. The electronic device of claim 1, wherein the at least one objectcomprises at least one of a content title, an image title, a text icon,an image icon, a text content, an image content, a link content, and aninput area which are displayed on the content page.
 5. The electronicdevice of claim 1, wherein the instruction text is displayed in atooltip format.
 6. The electronic device of claim 1, comprising amicrophone configured to receive an audio signal from a user, whereinthe processor is configured to: recognize a voice from the audio signalfrom the microphone; determine whether a text corresponding to therecognized voice is related to at least one object in a content page;and select the related object when the text is related to the at leastone object.
 7. The electronic device of claim 6, wherein the processoris configured to perform an operation matching the selected object. 8.The electronic device of claim 6, wherein the processor is configured tosupport voice input when the selected object is an input area.
 9. Theelectronic device of claim 6, wherein, when the text corresponding tothe recognized voice is related to a plurality of objects in the contentpage, the processor is configured to: assign an index text to theplurality of related objects; and configure a content page such that theindex text is located corresponding to the plurality of related objectsto display the content page on the display.
 10. The electronic device ofclaim 6, wherein, when there is an object related to a text languagedifferent from a text language corresponding to the recognized voice,the processor is configured to: assign an instruction text to the objectrelated to the different text language; and configure a content pagesuch that the instruction text is located corresponding to the objectrelated to the different text language to display the content page onthe display.
 11. An operating method of an electronic device, the methodcomprising: obtaining content comprising at least one object;distinguishing the at least one object within the content; displaying aninstruction text corresponding to a non-text object among the at leastone object; and selecting the non-text object corresponding to theinstruction text when a voice command corresponding to the instructiontext is input.
 12. The method of claim 11, further comprising: analyzinga resource of the content; distinguishing the at least one object withinthe content on the basis of an analysis result; assigning theinstruction text to a non-text object when the at least one objectcomprises the non-text object; and configuring and displaying a contentpage such that the instruction text is located corresponding to thenon-text object.
 13. The method of claim 11, further comprising:receiving an audio signal from a user; recognizing a voice from thereceived audio signal; determining whether a text corresponding to therecognized voice is related to at least one object in a content page;and selecting the related object when the text is related to the atleast one object.
 14. The method of claim 13, wherein, when the textcorresponding to the recognized voice is related to a plurality ofobjects in the content page, the method further comprises assigning anindex text to the plurality of related objects, and configuring anddisplaying a content page such that the index text is locatedcorresponding to the plurality of related objects; and when there is anobject related to a text language different from a text languagecorresponding to the recognized voice, the method further comprisesassigning an instruction text to the object related to the differenttext language, and configuring and displaying a content page such thatthe instruction text is located corresponding to the object related tothe different text language.
 15. A computer recording medium that storesan instruction that is executed by at least one processor and isreadable by a computer, the instruction being configured to perform:obtaining content comprising at least one object; distinguishing the atleast one object within the content; displaying an instruction textcorresponding to a non-text object among the at least one object; andselecting the non-text object corresponding to the instruction text whena voice command corresponding to the instruction text is input.